Communication apparatus, communication system, and data communication method

ABSTRACT

A communication configuration capable of acquiring communication data within an image without need of a high precision synchronization process is realized. A transmission apparatus has a projector outputting an image, and an output image generation section generating the image output from the projector. The output image generation section generates a communication data image that records communication data, and the projector outputs a viewing image and the communication data image by setting an output time period of the communication data image to be longer than an output time period of each of sub-frame images that configure the viewing image. A receiving apparatus detects an event which is a luminance change equal to or greater than a prescribed threshold, receives input event information including a pixel position and occurrence time of an event occurrence pixel, detects a communication data image contained in the projected image on the basis of an event occurrence interval, and acquires communication data from the communication data image.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Phase of International PatentApplication No. PCT/JP2018/012521 filed on Mar. 27, 2018, which claimspriority benefit of Japanese Patent Application No. JP 2017-089130 filedin the Japan Patent Office on Apr. 28, 2017. Each of theabove-referenced applications is hereby incorporated herein by referencein its entirety.

TECHNICAL FIELD

The present disclosure relates to a communication apparatus, acommunication system, a data communication method, and a program. Morespecifically, the present disclosure relates to a communicationapparatus, a communication system, a data communication method, and aprogram for holding communication based on a change in light difficultfor a person to perceive.

BACKGROUND ART

There is a technique for embedding information imperceptible to a personinto an image and transmitting the information and holding communicationwhile projecting a viewing image for a person using a projector. As arepresentative example, there is known a communication techniqueemploying a projector that uses a DMD (Digital Micromirror Device) and aphotodetector.

A communication configuration using the DMD is described in, forexample, PTL 1 (Japanese Patent Laid-Open No. Hei 05-224644), NPL 1(Kitamura, “Position-Dependent Visible Light Communication: ProjectingMeta-Media Data along with Visual Images,” FIT2006 (5th Forum onInformation Technology), 2006), and NPL 2 (Kimura, “Study related tointeraction with video using visible light communication projector”(master's thesis, Department of Information and CommunicationEngineering, Graduate School of Information Science and Technology, TheUniversity of Tokyo, 2008)).

For example, in communication using the DMD, the DMD is subjected tomirror control (on/off) at a high speed (a high frequency) exceedinghuman perceptive characteristics for a viewing image projected using aprojector and communication information is embedded into the viewingimage.

A projected image by the projector is made by a photodetector capable ofdetecting luminance changes due to high-speed on/off switchover ofmirrors. Analyzing the luminance changes detected by the photodetectormakes it possible to analyze signals that configure the communicationinformation.

It is noted that, for increasing a data volume of the communicationinformation using the projected image by the projector, it is effective,for example, to record data in each of a plurality of regions of oneimage and to read these pieces of data in parallel.

However, to acquire the information in parallel from the image regions,an image capturing system synchronized with the projector that isprojected image projecting side is required. Furthermore, a computingdevice with advanced functionality for processing high frame rate imagesis required, disadvantageously resulting an increase in the scale of anapparatus and a cost increase.

CITATION LIST Patent Literature

[PTL 1]

Japanese Patent Laid-Open No. Hei 05-224644

Non Patent Literature

[NPL 1]

Kitamura, “Position-Dependent Visible Light Communication: ProjectingMeta-Media Data along with Visual Images,” FIT2006 (5th Forum onInformation Technology), 2006

[NPL 2]

Kimura, “Study related to interaction with video using visible lightcommunication projector” (master's thesis, Department of Information andCommunication Engineering, Graduate School of Information Science andTechnology, The University of Tokyo, 2008)

SUMMARY Technical Problem

The present disclosure has been achieved in the light of, for example,the problems described above, and an object of the present disclosure isto provide a communication apparatus, a communication system, a datacommunication method, and a program capable of dispensing with asynchronization process for synchronizing a projector with a camera andtransmitting and communicating much information with a simple apparatusin a communication configuration using, for example, a DMD.

Solution to Problem

According to a first aspect of the present disclosure, there is provideda transmission apparatus including: a projector outputting an image; andan output image generation section generating the image output from theprojector. The output image generation section generates a communicationdata image that records communication data. The projector performs anoutput process for outputting a viewing image and the communication dataimage generated by the output image generation section, and outputs theviewing image and the communication data image by setting an output timeperiod of the communication data image to be longer than an output timeperiod of each of sub-frame images that configure the viewing image.

Furthermore, according to a second aspect of the present disclosure,there is provided a receiving apparatus including: an image acquisitionsection capturing a projected image by a projector, detecting an eventwhich is a luminance change equal to or greater than a prescribedthreshold, and outputting event information including a pixel positionand occurrence time of an event occurrence pixel; an event analysissection to which the event information is input and which detects acommunication data image contained in the projected image on the basisof an event occurrence interval; and a data analysis section acquiringcommunication data from the communication data image.

Moreover, according to a third aspect of the present disclosure, thereis provided a communication system including a transmission apparatusand a receiving apparatus. The transmission apparatus includes aprojector outputting an image, and an output image generation sectiongenerating the image output from the projector. The output imagegeneration section generates a communication data image that recordscommunication data. The projector performs an output process foroutputting a viewing image and the communication data image generated bythe output image generation section, and outputs the viewing image andthe communication data image by setting an output time period of thecommunication data image to be longer than an output time period of eachof sub-frame images that configure the viewing image. The receivingapparatus includes: an image acquisition section capturing a projectedimage by the projector, detecting an event which is a luminance changeequal to or greater than a prescribed threshold, and outputting eventinformation including a pixel position and occurrence time of an eventoccurrence pixel; an event analysis section to which the eventinformation is input and which detects a communication data imagecontained in the projected image on the basis of an event occurrenceinterval; and a data analysis section acquiring communication data fromthe communication data image.

Furthermore, according to a fourth aspect of the present disclosure,there is provided a data communication method executed by a transmissionapparatus. The transmission apparatus includes a projector outputting animage, and an output image generation section generating the imageoutput from the projector. The output image generation section generatesa communication data image that records communication data. Theprojector performs an output process for outputting a viewing image andthe communication data image generated by the output image generationsection, and outputs the viewing image and the communication data imageby setting an output time period of the communication data image to belonger than an output time period of each of sub-frame images thatconfigure the viewing image.

Moreover, according to a fifth aspect of the present disclosure, thereis provided a data communication method executed by a receivingapparatus, in which: an image acquisition section executes an imageacquisition process for capturing a projected image by a projector, fordetecting an event which is a luminance change equal to or greater thana prescribed threshold, and for outputting event information including apixel position and occurrence time of an event occurrence pixel; anevent analysis section executes an event analysis process for causingthe event information to be input to the event analysis section, and fordetecting a communication data image contained in the projected image onthe basis of an event occurrence interval; and a data analysis sectionexecutes a data analysis process for acquiring communication data fromthe communication data image.

Furthermore, according to a sixth aspect of the present disclosure,there is provided a program for causing a transmission apparatus toexecute a data transmission process. The transmission apparatus includesa projector outputting an image, and an output image generation sectiongenerating the image output from the projector. The program causes theoutput image generation section to generate a communication data imagethat records communication data. The program causes the projector toperform an output process for outputting a viewing image and thecommunication data image generated by the output image generationsection, and to output the viewing image and the communication dataimage by setting an output time period of the communication data imageto be longer than an output time period of each of sub-frame images thatconfigure the viewing image.

Moreover, according to a seventh aspect of the present disclosure, thereis provided a program for causing a receiving apparatus to execute adata receiving process, including: causing an image acquisition sectionto execute an image acquisition process for capturing a projected imageby a projector, for detecting an event which is a luminance change equalto or greater than a prescribed threshold, and for outputting eventinformation including a pixel position and occurrence time of an eventoccurrence pixel; causing an event analysis section to execute an eventanalysis process for causing the event information to be input to theevent analysis section, and for detecting a communication data imagecontained in the projected image on the basis of an event occurrenceinterval; and causing a data analysis section to execute a data analysisprocess for acquiring communication data from the communication dataimage.

It is noted that the programs of the present disclosure are those whichcan be provided to, for example, an information processing apparatus ora computer system capable of executing various programs/codes by astorage medium or a communication medium in a computer readable fashion.Providing such programs in the computer readable fashion enables theinformation processing apparatus or the computer system to realizeprocesses according to the programs.

Still other objects, features, and advantages of the present disclosurewill be readily apparent from more detailed description based on anembodiment of the present disclosure to be described later andaccompanying drawings. It is noted that a system means in the presentspecification a logical assembly configuration of a plurality ofapparatuses and is not limited to a system in which apparatuses withconfigurations are provided in the same casing.

Advantageous Effect of Invention

According to a configuration of one embodiment of the presentdisclosure, a communication configuration capable of acquiringcommunication data within an image without need of a high precisionsynchronization process is realized.

Specifically, a transmission apparatus has, for example, a projectoroutputting an image, and an output image generation section generatingthe image output from the projector. The output image generation sectiongenerates a communication data image that records communication data,and the projector outputs a viewing image and the communication dataimage by setting an output time period of the communication data imageto be longer than an output time period of each of sub-frame images thatconfigure the viewing image. A receiving apparatus detects an eventwhich is a luminance change equal to or greater than a prescribedthreshold, receives input event information including a pixel positionand occurrence time of an event occurrence pixel, detects acommunication data image contained in the projected image on the basisof an event occurrence interval, and acquires communication data fromthe communication data image.

With the present configuration, a communication configuration capable ofacquiring communication data within an image without need of a highprecision synchronization process is realized.

The advantages described in the present specification are given as anexample only, and the advantages are not limited to those described inthe present specification and may contain additional advantages.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an explanatory diagram of an outline of a communicationtechnique using a DMD (Digital Micromirror Device).

FIG. 2 is an explanatory diagram of an example of a configuration of aprojector using the DMD.

FIG. 3 is an explanatory diagram of an outline of a communicationtechnique using the DMD.

FIG. 4 is an explanatory diagram of the outline of the communicationtechnique using the DMD.

FIG. 5 is an explanatory diagram of an example of a configuration with ahigh-speed camera and data acquisition in communication using the DMD.

FIG. 6 is an explanatory diagram of an example of a synchronizationprocess for synchronizing exposure time of the projector with imagecapturing timing of the high-speed camera.

FIG. 7 is an explanatory diagram of an example in which the exposuretime of the projector is out of synchronization with the image capturingtiming of the high-speed camera and a lag is generated in the imagecapturing timing of the high-speed camera.

FIG. 8 is an explanatory diagram of an example of a configuration inwhich data embedded into normal viewing data can be acquired using ahigh-speed camera asynchronous with the projector.

FIG. 9 is an explanatory diagram of an example of a configuration of acommunication system of the present disclosure.

FIG. 10 is an explanatory diagram of an example of a projected imageonto a screen 100 and an example of an occurrence sequence of events(luminance changes).

FIG. 11 is an explanatory diagram of an example of setting a pixel valueby four sub-frames.

FIG. 12 is an explanatory flowchart of a process sequence executed by atransmission apparatus.

FIG. 13 is an explanatory flowchart of a process sequence executed by areceiving apparatus.

FIG. 14 is an explanatory diagram of an example of a hardwareconfiguration of a communication apparatus.

DESCRIPTION OF EMBODIMENTS

A communication apparatus, a communication system, a data communicationmethod, and a program according to the present disclosure will bedescribed hereinafter with reference to the drawings. It is noted thatdescription will be given in accordance with the following items.

1. Outline and problems of communication technique using DMD

2. Communication apparatus and communication process according topresent disclosure

3. Sequences of processes executed by transmission apparatus andreceiving apparatus

4. Example of hardware configuration of communication apparatus

5. Explanation of advantages derived from configuration of presentdisclosure

6. General overview of configuration of present disclosure

1. Outline and Problems of Communication Technique Using DMD

An outline and problems of a communication technique using a DMD(Digital Micromirror Device) will first be described with reference toFIG. 1 and subsequent drawings.

As described previously, there is a technique for embedding informationimperceptible to a viewer into a projected image and transmitting andcommunicating the information while projecting a viewing image for theviewer using a projector.

As a representative example, there is known a communication techniqueemploying a projector that uses a DMD (Digital Micromirror Device) and aphotodetector.

An outline of a communication process using the DMD will be describedwith reference to FIG. 1.

It is assumed, for example, that images projected using the projectorare images f1, 11 and f2, 12 depicted in FIG. 1, and these two imagesare projected onto a screen while alternately switching over the twoimages at a high speed.

In the image f1, 11, setting is made such that a region A is black and aregion B is white, and

in the image f2, 12, setting is made such that the region A is white andthe region B is black.

Such two images are projected onto the screen while being alternatelyswitched over at the high speed.

The switchover between the two images is performed at an intervalshorter than human visible light visual characteristics (approximately60 Hz).

As a result, an image observed by human eyes is an observation image 13that is an image obtained by summation averaging of the images f1, 11and f2, 12.

In the observation image 13, both of the regions A and B are gray imagesand a heart-shaped region is not at all recognized by human eyes.

While the figure indicates a heart-shaped frame by a dotted line withinthe observation image 13, this indicates that the observation image 13is the image obtained by summation of the images f1, 11 and f2, 12 and aperson is unable to recognize this heart-shaped region.

It is noted, however that it is sensed even by human eyes that theimages f, 11 and f2, 12 are alternately displayed if a switching speedbetween the images f1, 11 and f2, 12 is slow.

On the other hand, in a case in which the switching speed between theimages f1, 11 and f2, 12 is fast, it is not sensed by human eyes thatthe images f1, 11 and f2, 12 are alternately displayed.

The human visible light visual characteristics are approximately 60 Hz;thus, if an image switching frequency is equal to or higher than 60 Hz,a person is unable to recognize that the images f1, 11 and f2, 12 arealternately displayed.

As a device for switching over between the images f1, 11 and f2, 12 at ahigh speed equal to or higher than 60 Hz, there is known a projectorusing the DMD (Digital Micromirror Device).

An example of a configuration of the projector using the DMD (DigitalMicromirror Device) will be described with reference to FIG. 2.

FIG. 2 depicts a DMD 23 that receives light from a light source 21 andthat projects reflected light onto a screen 24.

The DMD 23 is a device on which several millions of microscopic mirrors25 each configured with, for example, an aluminum alloy approximately 20pm square are arranged, and which makes the mirrors individuallycontrollable at a high speed on the basis of a control signal input froma mirror control section 22.

A right side of FIG. 2 depicts “DMD cross-sectional enlarged view”depicting a cross-sectional configuration of the DMD 23.

In the “DMD cross-sectional enlarged view,” three mirrors a to c thatconfigure the DMD 23 are depicted.

An angle of the mirrors a and c differs from that of the mirror b.

The angle of the mirrors a and c is set to an angle at which incidentlight from the light source 21 is reflected in a screen direction.

With this setting, the light from the light source 21 is radiated ontothe screen 24.

On the other hand, the angle of the mirror b is set to an angle at whichthe incident light from the light source 21 is reflected onto anexterior in a direction other than the screen direction.

With this setting, the light from the light source 21 is not radiatedonto the screen 24.

In this way, the DMD 23 has a configuration capable of individuallycontrolling the microscopic mirrors.

It is assumed that the setting of the mirror for radiating the incidentlight from the light source 21 in the screen direction is “on-setting”and the setting of the mirror for not radiating the incident light fromthe light source 21 in the screen direction is “off-setting.”

The DMD 23 can switch over the mirror setting angles at a high speed of,for example, 8 kHz.

Using such a DMD makes it possible to exercise output control over eachpixel that configures an image radiated onto the screen.

An example of a process for executing a switching process over betweenthe images f1, 11 and f2, 12 described with reference to FIG. 1 by DMDmirror control will be described with reference to FIG. 3.

FIG. 3 depicts the following images:

two images similar to those described with reference to FIG. 1, that is,

the images f1, 11 and f2, 12; and

the observation image 13 that is the image observed by high-speedswitching of these two images.

Furthermore, an example of the DMD mirror control is depicted in a lowerportion of FIG. 3.

In the example of the DMD mirror control depicted in the lower portionof FIG. 3,

time transition data regarding mirror setting in the region Acorresponding to a heart external region and the region B correspondingto a heart internal region is depicted.

In both of the regions A and B, off and on are repeated.

As described above, the on-setting is the setting of the mirror angle atwhich the light is radiated in the screen direction, and the off-settingis the setting of the mirror angle at which the light is not radiated inthe screen direction.

The image f1, 11 corresponds to an output image at a time of setting theregion A off and the region B on.

On the other hand, the image f2, 12 corresponds to an output image at atime of setting the region A on and the region B off.

In this way, controlling the mirrors of the DMD per region makes itpossible to alternately output the images f1, 11 and f2, 12 onto thescreen.

By executing mirror on/off switch control at the high speed equal to orhigher than 60 Hz, the viewer who observes the screen observes only theobservation image 13 that is overall gray and is unable to sense theimages f1, 11 and f2, 12.

However, analyzing the image on the screen using the photodetector thatcan detect a luminance change at the speed equal to or higher than 60 Hzmakes it possible to detect the images f1, 11 and f2, 12.

Analyzing a detection signal of the photodetector makes it possible toacquire the heart-shaped image on the screen as communicationinformation.

In this way, using the photodetector that can detect a high-speedluminance change for the projected image by the projector makes itpossible to analyze signals that configure the communicationinformation.

To increase information acquired from the image, it is effective toadopt a configuration such that information is acquired in parallel fromimage regions that configure the projected image.

As depicted in FIG. 4, for example, an image 31 is divided into fourimage regions A to D and mirror control in each region is exercised bysetting a sequence as follows.

Region A=on→on→off→off,

Region B=off→off→on→on,

Region C=on→off→on→off, and

Region D=off→on→off→on.

It is assumed that the on/off switching is performed at the highfrequency equal to or higher than 60 Hz.

In each of the regions A to D, the sequence is set such that ‘on’ istwice and ‘off’ is twice, and the observation image visible to humaneyes is an image at the same luminance (gray) in the regions A to D as atotal and is observed as an image at a uniform luminance withoutdistinction among the regions A to D.

However, using the photodetector that can detect the high-speedluminance change makes it possible to obtain individual communicationinformation as follows for the regions A to D, respectively.

A=1100

B=0011

C=1010

D=0101

Nevertheless, to discriminate a signal for each region, it is necessaryto perform a process for detecting different high-speed luminancechanges that occur individually to the regions.

Examples of a configuration for realizing this process include aconfiguration with a high-speed camera.

An example of the configuration with the high-speed camera and anexample of acquiring data will be described with reference to FIG. 5 andsubsequent drawings.

FIG. 5 depicts a screen 31 that displays thereon a projected image 30 bythe projector, a high-speed camera 32 that captures the projected image30 on this screen 31, and imaging elements 33 of the high-speed camera32.

The high-speed camera 32 is a camera capable of setting short exposuretime and capturing an image. The high-speed camera 32 can set theexposure time to, for example, approximately ( 1/100) sec to ( 1/10000)sec.

Capturing the projected image 30 on the screen 31 using such ahigh-speed camera 32 enables communication data embedded into theprojected image 30 to be acquired as data regarding each regioncorresponding to each of resolution levels of the imaging elements ofthe high-speed camera 32.

However, to insert a communication image between image frames of normalviewing images and acquire the communication data from the communicationdata image, the high-speed camera 32 needs to capture a communicationdata image, that is, to perform an exposure process at instantaneouscommunication data image insertion timing. In other words, it isnecessary to synchronize image output time of the projector withexposure timing of the high-speed camera.

This synchronization process will be described with reference to FIG. 6.

FIG. 6 depicts a sequence of images that configure the projected image30 by the projector.

Time (t) passes from left to right.

During time t0 to t1, a screen projected image is a viewing image 41.

During time t1 to t2, a screen projected image is a communication dataimage 42.

During time t2 to t3, a screen projected image is a viewing image 43.

During time t3 to t4, a screen projected image is a viewing image 44.

It is noted that the images are switched over at a frequency equal to orhigher than 60 Hz.

It is assumed, for example, that the images are switched over at a framerate of 240 f/s, that is, at intervals of ( 1/240) seconds.

By switching over the images at such a high speed, the viewer graspsonly the viewing images as the observation image without notice of thecommunication data image 42 inserted during the time t1 to t2.

However, the high-speed camera 32 required to acquire the communicationdata needs to capture the communication data image 42 inserted duringthe time t1 to t2.

In other words, as depicted in FIG. 6, to insert the communication dataimage between the normal viewing images and to reliably acquire thecommunication data from this communication data image 42, it isnecessary to synchronize output timing at which the communication dataimage 42 is output from the projector with the exposure timing of thehigh-speed camera.

In the example depicted in FIG. 6, the images are switched over at the (1/240) sec intervals and quite high precision synchronization control isrequired.

FIG. 7 depicts an example in which the exposure time of the projector isout of synchronization with the image capturing timing of the high-speedcamera and a lag is generated in the image capturing timing of thehigh-speed camera.

As depicted in FIG. 7, when exposure of the high-speed camera 32 startsbefore the time t1 at which output of the communication data image 42starts, the high-speed camera 32 acquires image data having pixel valuesobtained by adding up the viewing image 41 and the communication dataimage 42.

If the high-speed camera 32 performs out-of-synchronization imagecapturing, configuration pixel values of the communication data image 42cannot be accurately acquired and it is impossible to analyze transmitdata.

In this way, to insert the communication data image between the normalviewing images and to acquire the communication data from thecommunication data image, it is necessary to accurately synchronize theimage output time of the projector with the image capturing timing ofthe high-speed camera. This requires a high precision controlconfiguration to be provided, disadvantageously resulting in a costincrease.

As opposed to the configuration required to perform such a highprecision synchronization process, there is proposed a configurationthat can acquire data embedded into normal viewing data using ahigh-speed camera asynchronous with a projector.

This configuration will be described with reference to FIG. 8.

As depicted in FIG. 8, a projected image 50 by the projector is dividedinto a plurality of sub-regions.

In an example depicted in FIG. 8, the projected image 50 is divided intofour sub-regions A to D.

Communication signals shifted in phase are embedded into thesub-regions, respectively.

A specific example of embedding the signals shifted in phase is depictedin an input signal example (1) in a middle stage of FIG. 8.

In the input signal example (1) in the middle stage of FIG. 8, signalsequences are set for the regions A, B, C, and D, respectively, and timepasses from left to right.

‘on’ indicates a state in which light is radiated on the screen and‘off’ indicates a state in which the light is not radiated on thescreen.

To the regions A to D, on/off pattern signals at the same periodcorresponding to transmit data are input. However, these signals areslightly shifted in phase among the regions.

The high-speed camera captures the projected image 50.

If the exposure timing of the camera is synchronized with any of thesignals for the regions A to D, the signal can be read.

In this way, by embedding the same transmission signal into eachsub-region while slightly shifting the phase, a probability ofsucceeding in reading any one of the transmission signals increases andthe high precision exposure time period synchronization process isunnecessary.

However, this method has problems that information can be reconstructedonly from part of the projected image by the projector and informationcan be embedded only into a still image.

Moreover, the processes described with reference to FIGS. 5 to 8 areeach based on the configuration with the high-speed camera, and it isnecessary to transfer the captured image by the camera to an informationprocessing apparatus such as a PC executing data analysis. This, inturn, requires a communication function having a sufficient band widthas a communication configuration for this transfer process. Moreover,the information processing apparatus such as the PC performing imageanalysis disadvantageously needs a configuration such as a highlyadvanced processor or a high-speed memory having a high computingcapability for performing an image process at a high frame rate.

2. Communication Apparatus and Communication Process According toPresent Disclosure

An example of a configuration of the present disclosure that hasovercome the problems described above will next be described.

A communication system according to the present disclosure does not needto perform a high precision synchronization process between a datatransmitting side that outputs an image having communication datarecorded therein using a projector or the like and a data receiving sidethat acquires the communication data from the image.

Furthermore, it is possible to mitigate a data analysis load of the datareceiving side.

FIG. 9 depicts an example of a configuration of the communication systemaccording to the present disclosure.

The communication system depicted in FIG. 9 is configured with atransmission apparatus 110 having a projector that projects an imagecontaining transmit data onto a screen 100, and a receiving apparatus120 having an image acquisition section that captures the imageprojected onto the screen 100.

The transmission apparatus 110 has a viewing image supply section 111, acommunication data supply section 112, an output image generationsection 113, a projector 114, and a control section 115.

On the other hand, the receiving apparatus 120 has an image acquisitionsection 121, an event analysis section 122, a data analysis section 123,and a control section 125.

A configuration and a process of the transmission apparatus 110 willfirst be described.

The viewing image supply section 111 supplies a viewing image for aperson, for example, a normal viewing image such as a still image or amotion video content of a movie or the like to the output imagegeneration section 113.

The communication data supply section 112 supplies communication data tobe embedded into the viewing image to the output image generationsection 113. This communication data is projected onto the screen 100 asa communication data image by the projector 114. It is noted, however,that projection time of the communication data image projected onto thescreen 100 is momentary and cannot be recognized by viewing by generalhuman eyes. The communication data image can be acquired only by a dataanalysis process using a configuration of the receiving apparatus 120.

The viewing image is input to the output image generation section 113from the viewing image supply section 111, and the communication data isinput thereto from the communication data supply section 112.

In a case in which the communication data is not input to the outputimage generation section 113 from the communication data supply section112, then the output image generation section 113 outputs the viewingimage input from the viewing image supply section 111 to the projector114 as it is, and the projector 114 projects the viewing image onto thescreen.

On the other hand, in a case in which the communication data is input tothe output image generation section 113 from the communication datasupply section 112, the output image generation section 113 generates acommunication data image on the basis of the input communication data.

The output image generation section 113 generates two communication dataimages using the viewing image immediately preceding output of thecommunication data image to the screen 100.

The output image generation section 113 outputs the generated twocommunication data images to the projector 114, and the projector 114projects these two communication data images onto the screen.

A specific process for these two communication data images will bedescribed later.

The projector 114 is a projector of a DMD scheme described above.

In other words, the projector 114 controls execution/stop of output ofthe image to the screen 100 per pixel by mirror control (on/off control)per pixel and projects the image onto the screen 100.

The projector 114 exercises the mirror control based on the image dataconverted by the output image generation section 113.

The control section 115 executes operation control over the viewingimage supply section 111, the communication data supply section 112, theoutput image generation section 113, and the projector 114, datatransfer control between the processing sections, and the like.

As for the DMD mirror control of the projector 114 depicted in thefigure, the projector 114 may include a mirror control section thereinor the control section 115 may execute the mirror control.

On the other hand, the receiving apparatus 120 has the image acquisitionsection 121, the event analysis section 122, the data analysis section123, and the control section 125.

The image acquisition section 121 is configured with an event camerathat captures an image projected onto the screen 100.

The event camera is a camera that has functions to perform an imagecapturing process, and to independently output, for each pixel to whicha luminance change (referred to as “event”) equal to or greater than aprescribed threshold occurs, a position (x,y) and luminance changeoccurrence time t of the pixel.

The event camera has a time resolution, for example, equal to or lowerthan 10 us, and can detect a luminance change of the screen projectedimage by the mirror control of the projector 114 of the DMD scheme inthe transmission apparatus 110 per pixel.

The event camera that is the image acquisition section 121 acquiresluminance change information (pixel position (x,y) and occurrence time(t)) of the image projected onto the screen 100 along with the capturedimage.

The pixel to which the luminance change equal to or greater than thethreshold occurs will be referred to as “event occurrence pixel,” andthe information including the pixel position and the occurrence time ofthe event occurrence pixel will be referred to as “event information.”

The image acquisition section 121 outputs the event informationincluding the luminance change information (pixel position (x,y) andoccurrence time (t)) to the event analysis section 122, and outputs thecaptured image data to the data analysis section 123.

It is noted that the captured image data output from the imageacquisition section 121 to the data analysis section 123 may only be thecommunication data image that stores therein the communication datadiscriminated on the basis of an event analysis process performed by theevent analysis section 122 and an image necessary to acquire thecommunication data from this communication data image.

A specific example of a communication data analysis process performed bythe data analysis section 123 will be described later.

The event information including the luminance change information (pixelposition (x,y) and occurrence time (t)) is input to the event analysissection 122 from the image acquisition section 121, and the eventanalysis section 122 detects an insertion position of the communicationdata image, that is, what image frame is a communication data imageframe on the basis of this input information.

Upon detecting the communication data image frame, the event analysissection 122 causes the image acquisition section 121 to output thecommunication data image and the image data necessary to a communicationdata acquisition process from this communication data image to the dataanalysis section 123.

The communication data image that records therein the communication dataidentified by the event analysis section 122 on the basis of theluminance change information (pixel position (x,y) and occurrence time(t)), and the image necessary to acquire the communication data fromthis communication data image are input to the data analysis section 123from the image acquisition section 121, and the data analysis section123 executes data analysis based on the input communication data imageand acquires communication data 130.

A specific process will be described later.

The control section 125 executes operation control over the imageacquisition section 121, the event analysis section 122, and the dataanalysis section 123, data transfer control between the processingsections, and the like.

An example of the projected image onto the screen 100 and an example ofan event (luminance change) occurrence sequence will be described withreference to FIG. 10.

FIG. 10 depicts an output sequence of four image frames that are imageframes A to D configuring the projected image onto the screen 100.

The image frame A that is an output image during time t0 to t1 is aviewing image 201.

The image frame B that is an output image during time t1 to t2 is acommunication data image 211.

The image frame C that is an output image during time t2 to t3 is acommunication data image 211.

The image frame D that is an output image during time t3 to t4 is aviewing image 202.

It is noted that an output time period of one image frame can bevariously set. However, it is necessary to set the output time period totime at such a level that the communication data image inserted betweenthe viewing images is unrecognizable.

An image frame switching frequency is assumed as 240 Hz, hereinafter, asan example. In this case, one frame is switched over to another at aninterval of ( 1/240) sec.

The viewing images 201 and 202 are each configured with a combination ofsub-frames (bit planes) including binary images to be projected by theprojector 114 of the DMD scheme.

FIG. 10 depicts an example in which the viewing image 201 is configuredwith four sub-frames 201 a to 201 d.

Sequentially outputting these four sub-frames 201 a to 201 d makes itpossible to output an image for which each pixel is set to a four-bitpixel value (0000 to 1111) with 16-level gray scale.

As previously described with reference to FIG. 2 and the like, theprojector 114 of the DMD scheme only switches over between an on-state(1) in which light is radiated onto the screen and an off-state (0) inwhich light is not radiated onto the screen. To express light and shadeand the like with the binary data, it is necessary to output acombination of sub-frames including a plurality of binary images.

It is noted that an image display configuration of this DMD scheme isdescribed in PTL 1 (Japanese Patent Laid-Open No. Hei 05-224644).

The four sub-frames 201 a to 201 d that configure the viewing image 201differ in output time, the output time of the sub-frame 201 a is set theshortest, and that of the sub-frame 201 d is set the longest.

The sub-frame 201 d having the longest output time is the bit plane thatspecifies a bit value of an MSB in each of the four-bit pixel values(0000 to 1111), while the sub-frame 201 a having the shortest outputtime is the bit plane that specifies a bit value of an LSB in each ofthe four-bit pixel values (0000 to 1111).

An example of setting the pixel value by these four sub-frames will bedescribed with reference to FIG. 11.

FIG. 11 is an explanatory diagram of an example of setting a pixel valueby the sub-frames 201 a to 201 d corresponding to the setting of each ofthe four-bit pixel values (0000 to 1111).

For example, an entry (1) depicted in FIG. 11 is an entry that indicatesthe setting of a pixel value by the sub-frames 201 a to 201 d in a casein which an output pixel value is (1111).

In this case, the sub-frames 201 a to 201 d are all set on, that is, thesub-frames 201 a to 201 d are all set such that light is radiated ontothe screen.

With this setting, this pixel value is set to the pixel value of (1111)as the four-bit pixel value.

An entry (2) depicted in FIG. 11 is an entry that indicates the settingof a pixel value by the sub-frames 201 a to 201 d in a case in which anoutput pixel value is (1110).

In this case, the sub-frames 201 a to 201 c are set on and only thesub-frame 201 d is set off.

With this setting, this pixel value is set to the pixel value of (1110)as the four-bit pixel value.

Furthermore, an entry (3) depicted in FIG. 11 is an entry that indicatesthe setting of a pixel value by the sub-frames 201 a to 201 d in a casein which an output pixel value is (1101).

In this case, the sub-frames 201 a, 201 b, and 201 d are set on and onlythe sub-frame 201 c is set off.

With this setting, this pixel value is set to the pixel value of (1101)as the four-bit pixel value.

This similarly applies to subsequent entries, and an entry (16) depictedin FIG. 11 is an entry that indicates the setting of a pixel value bythe sub-frames 201 a to 201 d in a case in which an output pixel valueis (0000).

In this case, the sub-frames 201 a to 201 d are all set off.

With this setting, this pixel value is set to the pixel value of (0000)as the four-bit pixel value.

In this way, adjusting the output time and performing an output processsuch that each of the four sub-frames corresponds to output of each bitvalue in the four-bit pixel values (0000 to 1111) enables the projectorof the DMD scheme to output a multi-gray-scale image.

The four sub-frames 201 a to 201 d that configure the viewing image 201depicted in FIG. 10 are output while being switched over sequentiallyand continuously within the output time (for example, ( 1/240) sec) ofone frame (frame A).

Therefore, output time of each sub-frame is shorter than output time ofthe frame, for example, approximately ( 1/300) to ( 1/4000) sec.

This similarly applies to the four sub-frames 202 a to 202 d thatconfigure the viewing image 202 depicted in FIG. 10, and they are outputwhile being switched over sequentially and continuously within theoutput time (for example, ( 1/240) sec) of one frame D.

Therefore, output time of each sub-frame is shorter than output time ofthe frame, for example, approximately ( 1/300) to ( 1/4000) sec.

While FIGS. 10 and 11 depict the example in which each pixel value ofthe viewing image is the four-bit pixel value, it is also possible tooutput a multi-gray-scale image having an eight-bit pixel value byusing, for example, eight sub-frames.

Furthermore, by setting output of an RGB limited color range as a lightsource and setting sub-frames in response to output timing of eachcolor, it is possible to output a color image.

With reference back to FIG. 10, an example of the projected image ontothe screen 100 and an example of an event (luminance change) occurrencesequence will be described.

As previously described, FIG. 10 depicts the output sequence of fourframes that are the image frames A to D configuring the projected imageonto the screen 100.

The image frame A that is the output image during the time t0 to t1 isthe viewing image 201.

The image frame B that is the output image during the time t1 to t2 isthe communication data image 211.

The image frame C that is the output image during the time t2 to t3 isthe communication data image 212.

The image frame D that is the output image during the time t3 to t4 isthe viewing image 202.

Out of these image frames,

the image frames A and D are the viewing images each generated by theoutput process of the plurality of sub-frames.

By contrast, the image frames B and C are the communication data imageswithout sub-frames.

The communication data images 211 and 212 are generated by the outputimage generation section 113 in the transmission apparatus 110 depictedin FIG. 9.

A specific example of a generation process for generating acommunication data image executed by the output image generation section113 will be described below.

It is assumed that (I_B(x,y)) is a pixel value an image of the imageframe B at a pixel position (x,y), and that

(I_C(x,y)) is a pixel value of an image of the image frame C at thepixel position (x,y).

At this time, a communication data image (I_B(x,y)) of the image frame Band a communication data image (I_C(x,y)) of the image frame C are eachassumed as an image generated by the following (Equations 1) whilecommunication data is defined as (I_send(x,y)).I_B(x,y)=I_last(x,y) (I_send(x,y)=0)I_B(x,y)=˜I_last(x,y) (I_send(x,y)=1)I_C(x,y)=˜I_B(x,y)   (Equations 1)

In (Equations 1),

I_send(x,y) is the communication data, one-bit communication data can beset for a position of each configuration pixel of an image, and piecesof data at a bit length equal to the number of configuration pixels ofone image frame can be transmitted simultaneously in parallel using theimage frame.

I_last(x,y) is a pixel value of each pixel (x,y) of a last sub-frame ofthe viewing image immediately preceding output of the image frames B andC.

In the example depicted in FIG. 10, the last sub-frame of the viewingimage 201 is the sub-frame 201 d, and I_last(x,y) is the pixel value ofthis sub-frame 201 d.

(˜I(x,y)) means inversion of a pixel value I(x,y) of an image frame I.

In a case of the pixel value I(x,y)=0, ˜I(x,y)=1

In a case of the pixel value I(x,y)=1, ˜I(x,y)=0.

(Equations 1) mean that the pixel value of the image frame B is set asfollows.

In a case in which a value of the communication data at the pixelposition (x,y) is 0,I_B(x,y)=I_last(x,y).

In other words, the pixel value of the pixel (x,y) of the last sub-framein the viewing image immediately preceding the output of the image frameB is output as it is.

On the other hand, in a case in which the communication data at thepixel position (x,y) is 1,I_B(x,y)=˜I_last(x,y).

In other words, the pixel value of the pixel (x,y) of the last sub-framein the viewing image immediately preceding the output of the image frameB is inverted and an inverse value is output.

Moreover, (Equations 1) mean that the pixel value of the image frame Cis set as follows.I_C(x,y)=˜I_B(x,y)

In other words, the pixel value of the image frame C at the pixelposition (x,y) is a value obtained by inverting the pixel value of theimage frame B at the pixel position (x,y).

The output image generation section 113 in the transmission apparatus110 depicted in FIG. 9 generates the image frames B and C, that is, thecommunication data images 211 and 212 depicted in FIG. 10 by the processdescribed above.

The projector 114 projects the two communication data images 211 and 212generated by the output image generation section 113 in the transmissionapparatus 110 onto the screen 100 as two consecutive image frames.

As can be understood from (Equations 1), the pixel values of the pixelsof the image frame B and those of the corresponding pixels of the imageframe C are set to be inverted.

Therefore, average pixel values of the two image frames, that is, animage with average pixel values set over the entire image is set to bemomentarily inserted between the viewing images (in a two-frame outputtime period).

The output time period of the two consecutively output image frames isas follows in a case in which the image frames at the switchingfrequency of, for example, 240 Hz are output as described above.2×( 1/240) sec=( 1/120) sec

This output time period corresponds to a display time period equal to orshorter than the human visual characteristics of ( 1/60) sec, and aviewer can view the viewing images without notice of the insertion ofthe two communication data image frames.

A communication data acquisition process performed by the receivingapparatus 120 will next be described.

As can be understood from the sequence diagram depicted in FIG. 10, theviewing images 201 and 204 for the viewer to view are each configuredwith the plurality of sub-frames, and most of the projected image hasluminance changes whenever the sub-frame is switched.

Therefore, when the event camera that is the image acquisition section121 in the receiving apparatus 120 captures the projected image, manyevents are detected in one frame a plurality of times within output timeperiods of the image frames A and D that are output time periods of theviewing images 201 and 204.

Downward arrows (i01 to i41) indicate event occurrences in “event” in amiddle stage depicted in FIG. 10.

In a case of assuming that the output time period of one frame is, forexample, ( 1/240) sec, the four sub-frames are switched and outputwithin ( 1/240) sec; thus, a plurality of event occurrences includingoutput of the initial sub-frame, that is, four event occurrences areconsecutively detected within one frame output time period of ( 1/240)sec.

Events i01 to i04 and events i31 to i34 depicted in FIG. 10 correspondto the event occurrences.

On the other hand, in an image frame output time period of each of theimage frames B and C in which the communication data images 211 and 212are output, no sub-frames are output.

Therefore, in the image frame output time periods of these communicationdata images, events (i11 and i21) occur only at times of initial outputof the image frames B and C.

In other words, the number of events occurring within the two imageframe output time periods (2×( 1/240) sec) is only two.

It is noted that at switching timing over between the image frames B andC, that is, at the time t2 depicted in FIG. 10, the event (i21) isdetected in all pixels.

In this way, an event occurrence interval in the output time periods ofthe image frames B and C in which the communication data images 211 and212 are output is equal to a frame period T.

In a case of assuming, for example, the frame period T as T=( 1/240)sec, the event occurrence interval is ( 1/240 sec).

The event analysis section 122 in the receiving apparatus 120 analyzesevent detection intervals detected by the image acquisition section(event camera) 121, and detects that projection of the communicationdata images has been executed at timing of confirming that the eventoccurrence intervals match the frame period=T for two consecutive times.

In the example depicted in FIG. 10, event occurrence timing in a timeperiod from the time t1 of start of the image frame B that is thecommunication data image 211 to the time t3 of start of the image frameD that is the viewing image 202 is the time t1, t2, and t3; thus, theevent occurrence intervals match the frame period=T for the twoconsecutive times.

In this case, the event analysis section 122 in the receiving apparatus120 determines that the transmitted image during the time t1 to t2 isthe communication data image storing therein the communication data.

It is noted that the communication data images 211 and 212 are acombination of data inverted from each other, and the event analysissection 122 can analyze the communication data using the onecommunication data image 211.

The event analysis section 122 causes the image acquisition section 121to output the communication data image 211 and the image data necessaryfor a communication data acquisition process for acquiring thecommunication data from this communication data image 211, which isspecifically the image data regarding the last sub-frame that configuresthe viewing image immediately preceding the output of the communicationdata image 211 to the data analysis section 123.

The communication data acquisition process is performed by the dataanalysis section 123 in the receiving apparatus 120 depicted in FIG. 9.

An example of the communication data acquisition process executed by thedata analysis section 123 will be described below.

As described above, if it is assumed that

the communication data is (I_send(x,y)), and that

the image data regarding the last sub-frame that configures the viewingimage immediately preceding the output of the communication data isI_last(x,y),

each configuration pixel value of the communication data image(I_B(x,y)) is set as a value according to the following Equations.I_B(x,y)=I_last(x,y) (I_send(x,y)=0)I_B(x,y)=˜I_last(x,y) (I_send(x,y)=1)

The data analysis section 123 in the receiving apparatus 120 depicted inFIG. 9 acquires the communication data (I_send(x,y)) by a comparisonprocess between the communication data image (I_B(x,y)) andcorresponding pixel values of the last sub-frame image (I_last(x,y))that configure the viewing image immediately preceding the output of thecommunication data.

In other words, the data analysis section 123 analyzes configuration bitvalues of the communication data by performing the followingdetermination process.

If I_B(x,y)=I_last(x,y), a configuration bit of the communicationdata=0, and

If I_B(x,y)=˜I_last(x,y), the configuration bit of the communicationdata=1.

In this way, with the configuration of the present disclosure, thereceiving apparatus 120 can determine at which timing the image storingtherein the communication data is transmitted (projected) by analyzingthe event occurrence intervals.

It is, therefore, unnecessary to perform the synchronization process,which is an essential requirement of the conventional apparatus, forsynchronizing the communication data transmission timing by theprojector in the transmission apparatus with the image capturing timingby the camera in the receiving apparatus.

Moreover, the image acquisition section (event camera) 121 in thereceiving apparatus 120 captures an image of each image frame along withevent detection and it is sufficient that, only in a case in which theevent occurrence intervals match the frame period=T for the twoconsecutive times, the image acquisition section (event camera) 121outputs the captured image of the first frame out of the two frames andthe sub-frame image immediately preceding the output of the two framesto the image analysis section 122, and the other images can be deleted.

Therefore, it is possible to reduce a required transfer data volume anda required memory capacity.

The example of using the projector of the DMD scheme as the projector114 in the transmission apparatus 110 has been described in the exampleof the configuration depicted in FIG. 9; however, the other device canbe also applied as long as the device is configured to execute theswitchover of the pixel values of the output image at the high speedsimilarly to the projector of the DMD scheme.

Moreover, the example of the configuration with the event camera as theimage acquisition section 121 in the receiving apparatus 120 has beendescribed in the example of the configuration depicted in FIG. 9;however, a device other than the event camera such as an ordinary cameramay be used as long as the device is capable of acquiring the luminancechange information similarly to the event camera.

3. Sequences of Processes Executed by Transmission Apparatus andReceiving Apparatus

Sequences of processes executed by the transmission apparatus 110 andthe receiving apparatus 120 depicted in FIG. 9 will next be describedwith reference to flowcharts depicted in FIGS. 12 and 13.

The sequence of the process executed by the transmission apparatus 110will first be described with reference to the flowchart depicted in FIG.12.

It is noted that the process according to a flow depicted in FIG. 12 isexecuted, for example, under control of a control section such as a CPUhaving a program execution function in accordance with a program storedin a storage section of the transmission apparatus 110.

The process in steps of the flow depicted in FIG. 12 will be describedin sequence.

(Step S101)

In Step S101, the transmission apparatus 110 sequentially outputssub-frame images that configure a viewing image.

This process is, for example, the output process for outputting theviewing image 201 described with reference to FIG. 10.

As described above, in the case in which the communication data is notinput to the output image generation section 113 in the transmissionapparatus 110 depicted in FIG. 9 from the communication data supplysection 112, then the output image generation section 113 outputs theviewing image input from the viewing image supply section 111 to theprojector 114 as it is, and the projector 114 projects the viewing imageonto the screen.

It is noted that the projector 114 performs image projection based onthe DMD scheme.

As described above, in an image projection process using the DMD, theprojector 114 changes the setting as to whether or not to radiate thelight onto the screen under on/off mirror control and sequentiallyoutputs the sub-frames including a plurality of bit planes in a case ofoutputting a multivalued image.

In Step S101, the transmission apparatus 110 sequentially outputs thesub-frame images that configure the viewing image.

Step S102)

Next, the transmission apparatus 110 determines whether communicationdata is present, and continues to output the viewing image in Step S101in a case of determining that the communication data is not present.

In a case of determining that the communication data is present, theprocess goes to Step S103.

(Step S103)

In the case of determining in Step S102 that the communication data ispresent, the transmission apparatus 110 generates a communication dataimage using the communication data and the image data regarding the lastsub-frame that configure the viewing image immediately preceding theoutput of the communication data in Step S103.

The generation process for generating this communication data image isexecuted by the output image generation section 113 in the transmissionapparatus 110 depicted in FIG. 9.

This image generation is executed in accordance with (Equations 1)previously described.

In other words, in the case of assuming that

the communication data is (I_send(x,y)), and that

the image data regarding the last sub-frame that configures the viewingimage immediately preceding the output of the communication data isI_last(x,y),

the output image generation section 113 generates the communication dataimage (I_B(x,y)) in accordance with the following Equations.I_B(x,y)=I_last(x,y) (I_send(x,y)=0)I_B(x,y)=˜I_last(x,y) (I_send(x,y)=1)(Step S104)

Next, the transmission apparatus 110 outputs the communication dataimage generated in Step S103 to the screen in Step S104.

The output time period is assumed as one frame time period (T).

(Step S105)

The transmission apparatus 110 then outputs an inverse image of thecommunication data image output in Step S104 to the screen in Step S105.

The output time period is assumed as one frame time period (T).

In other words, the output image generation section 113 generates andoutputs the communication data image inverse image (I_C(x,y)) of thecommunication data image (I_B(x,y)) in accordance with the followingEquation.I_C(x,y)=˜I_B(x,y)(Step S106)

Upon completion of output of the inverse image (I_C(x,y)) in Step S105,the transmission apparatus 110 performs an output process for outputtinga viewing image, that is, sequentially outputs sub-frame images thatconfigure the viewing image in Step S106.

Next, a sequence of the process executed by the receiving apparatus 120will be described with reference to the flowchart depicted in FIG. 13.

It is noted that the process according to a flow depicted in FIG. 13 isexecuted, for example, under control of a control section such as a CPUhaving a program execution function in accordance with a program storedin a storage section of the receiving apparatus 120.

The process in steps of the flow depicted in FIG. 13 will be describedin sequence.

(Step S201)

First, the receiving apparatus 120 executes capturing of the imageprojected onto the screen and detection of information regardingluminance changes (events) in Step S201.

This process is executed by the image acquisition section (event camera)121 in the receiving apparatus 120 depicted in FIG. 9.

As described above, the event camera is a camera that has the functionsto perform the image capturing process, and to output, for each pixel towhich the luminance change (event) equal to or greater than theprescribed threshold occurs, the event information including theposition (x,y) of the pixel and the luminance change occurrence time tindependently of the pixels.

(Step S202)

Next, the receiving apparatus 120 measures event occurrence intervals inStep S202.

This process is a process executed by the event analysis section 212depicted in FIG. 9.

As described above, the event information, that is, the luminance changeinformation (pixel position (x,y) and occurrence time (t)) is input tothe event analysis section 122 from the image acquisition section 121,and the event analysis section 122 detects the insertion position of thecommunication data image, that is, what image frame is a communicationdata image frame on the basis of this input information.

This detection process is executed on the basis of the event occurrenceinterval.

(Step S203)

The event analysis section 212 determines whether the event occurrenceintervals match the frame period (T) for the two consecutive times inStep S203.

As previously described with reference to FIG. 10, the event occurrenceinterval is shorter than the frame period (T) since a plurality ofsub-frames is switched over and output within one frame output timeperiod for the viewing image output frames.

However, since the output of the sub-frames is not executed in theoutput time periods of the two image frames, which are the communicationimage frame (I_B(x,y)) and the communication image inverse frame(I_C(x,y)) that is the inverse image of this image (I_B(x,y)) depictedin FIG. 10 as the communication image frame output time periods, theevent occurrence intervals are equal to the frame period (T) for the twoconsecutive times.

In a case of determining in Step S203 that the event occurrenceintervals match the frame period (T) for the two consecutive times, thenthe event analysis section 212 determines that the communication dataimages have been transmitted, and the process goes to Step S204.

On the other hand, in a case of determining in Step S203 that the eventoccurrence intervals do not match the frame period (T) for the twoconsecutive times, then the event analysis section 212 determines thatthe communication data images have not been transmitted, and the processreturns to Step S201.

(Step S204)

Next, the receiving apparatus 120 determines that the image within thetime period that is the initial frame period T as the communication dataimage and acquires the communication data image in Step S204.

This process is a process performed by the data analysis section 123depicted in FIG. 9 for acquiring the communication data image from theimage acquisition section 121.

In the example depicted in FIG. 10, the process corresponds to theprocess for acquiring the communication data image (I_B(x,y)) 211.

(Step S205)

Next, the receiving apparatus 120 analyzes the configuration pixelvalues of the communication data image and acquires the communicationdata in Step S205.

This process is a process executed by the data analysis section 123 inthe receiving apparatus 120 depicted in FIG. 9.

In the example depicted in FIG. 10, the communication data image(I_B(x,y)) transmitted during the time t1 to t2 is the image storingtherein the communication data, and the data analysis section 123 canacquire the communication data by analyzing the pixel values of thiscommunication data image 211.

As described above, if it is assumed that

the communication data is (I_send(x,y)), and that

the image data regarding the last sub-frame that configures the viewingimage immediately preceding the output of the communication data isI_last(x,y),

each configuration pixel value of the communication data image(I_B(x,y)) is set as a value according to the following Equations.I_B(x,y)=I_last(x,y) (I_send(x,y)=0)I_B(x,y)=˜I_last(x,y) (I_send(x,y)=1)

The data analysis section 123 in the receiving apparatus 120 depicted inFIG. 9 acquires the communication data (I_send(x,y)) by a comparisonprocess between the communication data image (I_B(x,y)) andcorresponding pixel values of the last sub-frame image (I_last(x,y))that configure the viewing image immediately preceding the output of thecommunication data (I_B(x,y)).

In other words, the data analysis section 123 analyzes eachconfiguration bit value of the communication data by performing thefollowing determination process.

If I_B(x,y)=I_last(x,y), the configuration bit of the communicationdata=0, and

If I_B(x,y)=˜I_last(x,y), the configuration bit of the communicationdata=1.

4. Example of Hardware Configuration of Communication Apparatus

A specific example of a hardware configuration of the transmissionapparatus 110 and the receiving apparatus 120 described with referenceto FIGS. 9 and 10 will next be described with reference to FIG. 14.

A CPU (Central Processing Unit) 301 functions as a data processingsection that executes various processes in accordance with a programstores in a ROM (Read Only Memory) 302 or a storage section 308. Forexample, the CPU 301 executes processes according to the sequencedescribed in the embodiment described above. A RAM (Random AccessMemory) 303 stores the program executed by the CPU 301, data, and thelike. The CPU 301, the ROM 302, and the RAM 303 are mutually connectedby a bus 304.

The CPU 301 is connected to an input/output interface 305 via the bus304, and an input section 306 including various switches, a keyboard, amouse, a microphone, a camera, and the like and an output section 307including a display, a speaker, and the like are connected to theinput/output interface 305.

In the case of the transmission apparatus 110, the output section 307includes the projector of the DMD scheme.

Furthermore, in the case of the receiving apparatus 120, the inputsection 306 includes the event camera.

Commands, status data, and the like are input to the CPU 301 from theinput section 306, and the CPU 301 executes various processes andoutputs process results to, for example, the output section 307.

The storage section 308 connected to the input/output interface 305 isconfigured with, for example, a hard disk and stores the programexecuted by the CPU 301 and various data. A communication section 309functions as a transmitting and receiving section for data communicationvia a network such as the Internet or a local area network, andcommunicates with external apparatuses.

A drive 310 connected to the input/output interface 305 drives aremovable medium 311 such as a magnetic disk, an optical disk, amagneto-optical disk or a semiconductor memory, which is, for example, amemory card, and executes recording or reading of data.

5. Explanation of Advantages Derived from Configuration of PresentDisclosure

A summary of advantages derived from the configuration of the presentdisclosure will be described.

(1) It is possible to transmit and receive communication data withoutbeing perceived by the viewer.

As previously described with reference to FIG. 10, the pixel values ofthe pixels of the image frame B corresponding to the communication dataimage 211 and those of the corresponding pixels of the image frame Ccorresponding to the communication data image 212 are set to beinverted.

Therefore, the average pixel values of the two image frames, that is,the image with average pixel values set over the entire image is set tobe momentarily inserted between the viewing images.

The output time period of the two consecutively output image frames isas follows.2×( 1/240) sec=( 1/120) sec

In other words, this is the display time period equal to or shorter thanthe human visual characteristics of ( 1/60) sec, and the viewer can viewthe viewing images without notice of the insertion of the twocommunication data image frames.

In other words, integrating the image frames B and C makes the image atthe uniform luminance. Thus, in a case in which the frame rate of theprojector sufficiently surpasses the human visible light perceptivecharacteristics, it is possible to transmit and receive thecommunication data image without being perceived by a person.

(2) It is possible to simultaneously receive information at a highspatial resolution equal to a resolution of the image acquisitionsection 121 in the receiving apparatus 120.

The communication data image is an image having communication data setto each pixel thereof.

In other words, as previously described as (Equations 1), if it isassumed that

the communication data is (I_send(x,y)), and that

the image data regarding the last sub-frame that configures the viewingimage immediately preceding the output of the communication data isI_last(x,y),

each configuration pixel value of the communication data image(I_B(x,y)) is set as the value according to the following Equations.I_B(x,y)=I_last(x,y) (I_send(x,y)=0)I_B(x,y)=˜I_last(x,y) (I_send(x,y)=1)

In this way, the communication data image (I_B(x,y)) is the image havingthe communication data set to each pixel thereof, so that it iseventually possible to simultaneously receive the information at thehigh spatial resolution equal to the resolution of the image acquisitionsection 121 in the receiving apparatus 120.

(3) It is possible to detect the communication data image and acquirethe communication data only from the event occurrence interval withoutconnection or synchronization between the projector and the camera.

As previously described, with the configuration of the presentdisclosure, the receiving apparatus 120 can determine at which timingthe image storing therein the communication data is transmitted(projected) by analyzing the event occurrence interval.

It is, therefore, unnecessary to perform the synchronization process,which is an essential requirement of the conventional apparatus, forsynchronizing the communication data transmission timing by theprojector in the transmission apparatus with the image capturing timingby the camera in the receiving apparatus.

In other words, it is possible to detect the communication data imageand acquire the communication data only from the event occurrenceinterval without connection or synchronization between the projector andthe camera.

(4) With the configuration of the present disclosure, it is sufficientthat only the data corresponding to the events for which the eventoccurrence intervals match the frame output interval (T) is sent to thesignal processing section; thus, a necessary band width is narrow,compared with a case of using the high-speed camera.

As previously described with reference to FIG. 9, the image acquisitionsection (event camera) 121 in the receiving apparatus 120 captures theimage of each image frame along with event detection and it issufficient that, only in the case in which the event occurrenceintervals match the frame period=T for the two consecutive times, theimage acquisition section (event camera) 121 outputs the captured image(=communication data image) of the first frame out of the two frames andthe sub-frame image immediately preceding the output of the two framesto the image analysis section 122, and the other images can be deleted.

Therefore, it is possible to reduce the required transfer data volumeand the required memory capacity.

(5) It is possible to acquire the communication data from eachcommunication data image with a simple process; thus, even a computingmachine having a low computing capability can perform the process.

With the configuration of the present disclosure, as previouslydescribed with reference to FIG. 9, the data analysis section 123 in thereceiving apparatus 120 acquires the communication data (I_send(x,y)) bythe comparison process between the communication data image (I_B(x,y))and the corresponding pixel values of the last sub-frame image(I_last(x,y)) that configure the viewing image immediately preceding theoutput of the communication data.

In other words, the data analysis section 123 analyzes eachconfiguration bit value of the communication data by performing thefollowing determination process.

If I_B(x,y)=I_last(x,y), the configuration bit of the communicationdata=0, and

If I_B(x,y)=˜I_last(x,y), the configuration bit of the communicationdata=1.

In this way, with the configuration of the present disclosure, it ispossible to acquire the communication data from each communication dataimage with a simple process; thus, even a computing machine having a lowcomputing capability can perform the process.

6. General Overview of Configuration of Present Disclosure

The embodiment of the present disclosure has been described so far indetail while referring to the specific embodiments. Nevertheless, it isobvious that a person ordinary skill in the art could make revision ofthe embodiments or find replacements therefor within the scope of thepresent disclosure. In other words, the present invention has beendisclosed in an illustrative form and should not be interpretedexclusively. Reference should be made to claims for the assessment ofthe scope of the present disclosure.

The technique disclosed in the present specification can be configuredas follows.

(1) A transmission apparatus including:

a projector outputting an image; and

an output image generation section generating the image output from theprojector, in which

the output image generation section generates a communication data imagethat records communication data,

the projector performs an output process for outputting a viewing imageand the communication data image generated by the output imagegeneration section, and

the projector outputs the viewing image and the communication data imageby setting an output time period of the communication data image to belonger than an output time period of each of sub-frame images thatconfigure the viewing image.

(2) The transmission apparatus according to (1), in which

the projector includes a projector to which a DMD (Digital MicromirrorDevice) is applied, and

the projector outputs the viewing image by switching over between aplurality of sub-frames each including a binary image within one imageframe output time period.

(3) The transmission apparatus according to (1) or (2), in which

the output image generation section generates a first communication dataimage upon setting each bit value by maintaining or inverting each bitvalue of a sub-frame of a viewing image immediately preceding output ofthe communication data image in response to each configuration bit valueof communication data.

(4) The transmission apparatus according to (3), in which

the output image generation section further generates a secondcommunication data image by inverting all configuration bit values ofthe first communication data image.

(5) The transmission apparatus according to (4), in which

the projector outputs two communication data images, which are the firstcommunication data image and the second communication data image, whileswitching over between the first communication data image and the secondcommunication data image per image frame output time period.

(6) A receiving apparatus including:

an image acquisition section capturing a projected image by a projector,detecting an event which is a luminance change equal to or greater thana prescribed threshold, and outputting event information including apixel position and occurrence time of an event occurrence pixel;

an event analysis section to which the event information is input andwhich detects a communication data image contained in the projectedimage on the basis of an event occurrence interval; and

a data analysis section acquiring communication data from thecommunication data image.

(7) The receiving apparatus according to (6), in which

the projected image by the projector includes a projected image by aprojector to which a DMD (Digital Micromirror Device) is applied,

the projected image contains two different images:

-   -   (a) a viewing image output while switching over between a        plurality of sub-frames each including a binary image within one        image frame output time period; and    -   (b) a communication data image continuously output for one image        frame output time period, and

the event analysis section determines whether or not the eventoccurrence interval matches the one image frame output time period, anddetects the communication data images contained in the projected image.

(8) The receiving apparatus according to (7), in which

the communication data image includes

-   -   (b1) a first communication data image continuously output for        one image frame output time period, and    -   (b2) a second communication data image that is a communication        data image continuously output for one image frame output time        period after output of the first communication data image, and        that is obtained by inverting all configuration bit values of        the first communication data image, and

the event analysis section determines whether or not event occurrenceintervals match the one image frame output time period for twoconsecutive times, and detects the communication data images containedin the projected image.

(9) The receiving apparatus according to (7) or (8), in which

the communication data image includes an image upon setting each bitvalue by maintaining or inverting each bit value of a last sub-frame ofthe viewing image immediately preceding output of the communication dataimage in response to each configuration bit value of communication data,and

the data analysis section acquires the communication data on the basisof the configuration bit values of the communication data image.

(10) The receiving apparatus according to any one of (7) to (9), inwhich

the communication data image includes an image upon setting each bitvalue by maintaining or inverting each bit value of a last sub-frame ofthe viewing image immediately preceding output of the communication dataimage in response to each configuration bit value of communication data,and

the data analysis section acquires the communication data by acomparison process between the configuration bit values of thecommunication data image and the bit values of the last sub-frame in theviewing image.

(11) A communication system including:

a transmission apparatus; and

a receiving apparatus, in which

the transmission apparatus includes

-   -   a projector outputting an image, and    -   an output image generation section generating the image output        from the projector,    -   the output image generation section generating a communication        data image that records communication data,    -   the projector performing an output process for outputting a        viewing image and the communication data image generated by the        output image generation section, and    -   the projector outputting the viewing image and the communication        data image by setting an output time period of the communication        data image to be longer than an output time period of each of        sub-frame images that configure the viewing image, and

the receiving apparatus includes

-   -   an image acquisition section capturing a projected image by the        projector, detecting an event which is a luminance change equal        to or greater than a prescribed threshold, and outputting event        information including a pixel position and occurrence time of an        event occurrence pixel,    -   an event analysis section to which the event information is        input and which detects a communication data image contained in        the projected image on the basis of an event occurrence        interval, and    -   a data analysis section acquiring communication data from the        communication data image.

(12) The communication system according to (11), in which

the projector in the transmission apparatus includes a projector towhich a DMD (Digital Micromirror Device) is applied, and

the projector outputs the viewing image by switching over between aplurality of sub-frames each including a binary image within one imageframe output time period.

(13) The communication system according to (12), in which

the projected image contains two different images:

-   -   (a) a viewing image output while switching over between a        plurality of sub-frames each including a binary image within one        image frame output time period; and    -   (b) a communication data image continuously output for one image        frame output time period, and

the event analysis section in the receiving apparatus determines whetheror not the event occurrence interval matches the one image frame outputtime period, and detects the communication data images contained in theprojected image.

(14) A data communication method executed by a transmission apparatus,in which

the transmission apparatus includes

-   -   a projector outputting an image, and    -   an output image generation section generating the image output        from the projector,    -   the output image generation section generating a communication        data image that records communication data,    -   the projector performing an output process for outputting a        viewing image and the communication data image generated by the        output image generation section, and    -   the projector outputting the viewing image and the communication        data image by setting an output time period of the communication        data image to be longer than an output time period of each of        sub-frame images that configure the viewing image.

(15) A data communication method executed by a receiving apparatus, inwhich

an image acquisition section executes an image acquisition process forcapturing a projected image by a projector, for detecting an event whichis a luminance change equal to or greater than a prescribed threshold,and for outputting event information including a pixel position andoccurrence time of an event occurrence pixel,

an event analysis section executes an event analysis process for causingthe event information to be input to the event analysis section, and fordetecting a communication data image contained in the projected image onthe basis of an event occurrence interval, and

a data analysis section executes a data analysis process for acquiringcommunication data from the communication data image.

(16) A program for causing a transmission apparatus to execute a datatransmission process, in which

the transmission apparatus includes

-   -   a projector outputting an image, and    -   an output image generation section generating the image output        from the projector,

the program causes the output image generation section to generate acommunication data image that records communication data, and

the program causes the projector to perform an output process foroutputting a viewing image and the communication data image generated bythe output image generation section, and to output the viewing image andthe communication data image by setting an output time period of thecommunication data image to be longer than an output time period of eachof sub-frame images that configure the viewing image.

(17) A program for causing a receiving apparatus to execute a datareceiving process, including:

causing an image acquisition section to execute an image acquisitionprocess for capturing a projected image by a projector, for detecting anevent which is a luminance change equal to or greater than a prescribedthreshold, and for outputting event information including a pixelposition and occurrence time of an event occurrence pixel;

causing an event analysis section to execute an event analysis processfor causing the event information to be input to the event analysissection, and for detecting a communication data image contained in theprojected image on the basis of an event occurrence interval; and

causing a data analysis section to execute a data analysis process foracquiring communication data from the communication data image.

Furthermore, a series of processes described in the specification can beexecuted by hardware, software or a combined configuration of thehardware and the software. In a case in which the processes are executedby the software, then a program recording a process sequence can beexecuted by installing the program in a memory within a computerincorporated into dedicated hardware, or can be executed by installingthe program in a general-purpose computer capable of executing variousprocesses. For example, the program can be recorded in a storage mediumin advance. The program can be executed by not only installing theprogram from the storage medium in the computer but also by receivingthe program via a network such as a LAN (Local Area Network) or theInternet and installing the received program in a recording medium suchas a hard disk embedded in the computer.

The various processes described in the specification may be executed notonly in time series in accordance with the description but also executedindividually or in parallel in response to a processing capability of anapparatus that executes the processes or as needed. Moreover, a systemmeans in the present specification a logical assembly configuration of aplurality of apparatuses and is not limited to a system in whichapparatuses with configurations are provided in the same casing.

INDUSTRIAL APPLICABILITY

As described so far, according to the configuration of one embodiment ofthe present disclosure, a communication configuration capable ofacquiring communication data within an image without need of a highprecision synchronization process is realized.

Specifically, a transmission apparatus has, for example, a projectorthat outputs an image; and an output image generation section thatgenerates the image output from the projector. The output imagegeneration section generates a communication data image that recordscommunication data, and the projector outputs a viewing image and thecommunication data image frame by setting an output time period of thecommunication data image to be longer than an output time period of eachof sub-frame images that configures the viewing image. A receivingapparatus detects an event which is a luminance change equal to orgreater than a prescribed threshold, receives input event informationformed from a pixel position and occurrence time of an event occurrencepixel, detects a communication data image contained in the projectedimage on the basis of an event occurrence interval, and acquirescommunication data from the communication data image.

With the present configuration, a communication configuration capable ofacquiring communication data within an image without need of a highprecision synchronization process is realized.

REFERENCE SIGNS LIST

-   21: Light source-   22: Mirror control section-   23: DMD-   24: Screen-   25: Mirror-   30: Projected image-   31: Screen-   32: High-speed camera-   33: Imaging element-   41, 43, 44: Viewing image-   42: Communication data image-   50: Projected image-   100: Screen-   110: Transmission apparatus-   111: Viewing image supply section-   112: Communication data supply section-   113: Output image generation section-   114: Projector-   115: Control section-   120: Receiving apparatus-   121: Image acquisition section-   122: Event analysis section-   123: Data analysis section-   125: Control section-   130: Communication data-   201, 202: Viewing image-   211, 212: Communication data image-   301: CPU-   302: ROM-   303: RAM-   304: Bus-   305: Input/output interface-   306: Input section-   307: Output section-   308: Storage section-   309: Communication section-   310: Drive-   311: Removable medium

The invention claimed is:
 1. A transmission apparatus, comprising: anoutput image generation section configured to: set each bit value of afirst communication data image based on each configuration bit value ofa plurality of configuration bit values of communication data, whereinthe set each bit value of the first communication data image is one ofequal or inverse of each bit value of a specific sub-frame of aplurality of sub-frames of a viewing image, and the specific sub-frameimmediately precedes output of the first communication data image;generate, based on the set each bit value of the first communicationdata image, the first communication data image that includes thecommunication data; and a projector configured to: set a first outputtime period of the first communication data image and a second outputtime period of each of the plurality of sub-frames of the viewing image,wherein the first output time period is longer than the second outputtime period; output each of the plurality of sub-frames of the viewingimage in the set second output time period; and output the firstcommunication data image in the set first output time period.
 2. Thetransmission apparatus according to claim 1, wherein the projectorincludes a DMD (Digital Micromirror Device), the projector is configuredto output the viewing image based on switch over between the pluralityof sub-frames within one image frame output time period, and each of theplurality of sub-frames includes a binary image.
 3. The transmissionapparatus according to claim 1, wherein the output image generationsection is further configured to generate a second communication dataimage based on inversion of the plurality of configuration bit values ofthe first communication data image.
 4. The transmission apparatusaccording to claim 3, wherein the projector is further configured tooutput, two communication data images that includes the firstcommunication data image and the second communication data image, basedon switch over between the first communication data image and the secondcommunication data image within one image frame output time period.
 5. Areceiving apparatus, comprising: an image acquisition section configuredto: capture a projected image, wherein the projected image is projectedby a projector that includes a DMD (Digital Micromirror Device), theprojected image includes a communication data image and a viewing image,the viewing image comprises a plurality of sub-frames, the projectorswitches between the plurality of sub-frames within one image frameoutput time period to output the viewing image, output of thecommunication data image is continuous for the one image frame outputtime period, each bit value of the communication data image is set basedon each configuration bit value of a plurality of configuration bitvalues of communication data, the set each bit value of thecommunication data image is one of equal or inverse of each bit value ofa last sub-frame of the plurality of sub-frames, and the last sub-frameimmediately precedes output of the communication data image; detect anevent which is a luminance change equal to or greater than a specificthreshold; and output event information including a pixel position andoccurrence time of an event occurrence pixel; an event analysis sectionconfigured to: receive the event information; determine a first matchbetween an event occurrence interval of the event and the one imageframe output time period; and detect the communication data imageincluded in the projected image based on the determination; and a dataanalysis section configured to acquire the communication data from thecommunication data image based on the plurality of configuration bitvalues of the communication data image.
 6. The receiving apparatusaccording to claim 5, wherein the communication data image includes afirst communication data image continuously output for the one imageframe output time period, and a second communication data image that iscontinuously output for the one image frame output time period afteroutput of the first communication data image, the second communicationdata image is inverse of the plurality of configuration bit values ofthe first communication data image, the image acquisition section isfurther configured to detect a plurality of events in which theluminance change is equal to or greater than the specific threshold, andthe event analysis section is further configured to: determine a secondmatch between event occurrence intervals of the plurality of events andthe one image frame output time period for two consecutive times; anddetect the communication data image included in the projected imagebased on the determination.
 7. The receiving apparatus according toclaim 5, wherein the data analysis section is further configured toacquire the communication data based on a comparison between theplurality of configuration bit values of the communication data imageand a plurality of bit values of the last sub-frame in the viewingimage.
 8. A communication system, comprising: a transmission apparatus;and a receiving apparatus, wherein the transmission apparatus includes:an output image generation section configured to generate acommunication data image that includes communication data, a projectorincluding a DMD (Digital Micromirror Device), wherein the projector isfurther configured to: set a first output time period of thecommunication data image and a second output time period of each of aplurality of sub-frames of a viewing image, wherein the first outputtime period is longer than the second output time period; output each ofthe plurality of sub-frames of the viewing image in the set secondoutput time period; output the communication data image in the set firstoutput time period; and output the viewing image based on switch overbetween the plurality of sub-frames of the viewing image within oneimage frame output time period, each of the plurality of sub-framesincludes a binary image, and the receiving apparatus includes: an imageacquisition section capturing configured to: capture a projected image,wherein  the projected image is projected by the projector,  theprojected image includes the viewing image and the communication dataimage, and  the communication data image is continuously output for theone image frame output time period; detect an event corresponding to aluminance change equal to or greater than a specific threshold; andoutput event information including a pixel position and occurrence timeof an event occurrence pixel; and an event analysis section isconfigured to: receive the event information; determine that an eventoccurrence interval of the event matches the one image frame output timeperiod; and detects the communication data image based on thedetermination; and a data analysis section configured to acquire thecommunication data from the communication data image.
 9. A datacommunication method, comprising: in a transmission apparatus: settingeach bit value of a communication data image based on each configurationbit value of communication data, wherein the set each bit value of thecommunication data image is one of equal or inverse of each bit value ofa specific sub-frame of a plurality of sub-frames of a viewing image,and the specific sub-frame immediately precedes output of thecommunication data image; generating, based on the set each bit value ofthe communication data image, the communication data image that includesthe communication data; setting a first output time period of thecommunication data image and a second output time period of each of theplurality of sub-frames, wherein the first output time period is longerthan the second output time period; output each of the plurality ofsub-frames of the viewing image in the set second output time period;and output the communication data image in the set first output timeperiod.
 10. A data communication method, comprising in a receivingapparatus: capturing a projected image, wherein the projected image isprojected by a projector, the projected image includes a communicationdata image and a viewing image, the viewing image comprises a pluralityof sub-frames, the projector switches between the plurality ofsub-frames within one image frame output time period to output theviewing image; output of the communication data image is continuous forthe one image frame output time period, each bit value of thecommunication data image is set based on each configuration bit value ofa plurality of configuration bit values of communication data, the seteach bit value of the communication data image is one of equal orinverse of each bit value of a last sub-frame of the plurality ofsub-frames, and the last sub-frame immediately precedes output of thecommunication data image; detecting an event corresponding to aluminance change equal to or greater than a specific threshold;outputting event information including a pixel position and occurrencetime of an event occurrence pixel; receiving the event information;determining that event occurrence interval of the event matches the oneimage frame output time period; detecting the communication data imagecontained in the projected image based on the determination; andacquiring the communication data from the communication data image basedon the plurality of configuration bit values of the communication dataimage.
 11. A non-transitory computer-readable medium having storedthereon computer-executable instructions that, when executed by atransmission apparatus, cause the transmission apparatus to executeoperations, the operations comprising: setting each bit value of acommunication data image based on each configuration bit value ofcommunication data, wherein the set each bit value of the communicationdata image is one of equal or inverse of each bit value of a specificsub-frame of a plurality of sub-frames of a viewing image, and thespecific sub-frame immediately precedes output of the communication dataimage; generating the communication data image that includes thecommunication data; setting a first output time period of thecommunication data image and a second output time period of each of theplurality of sub-frames, wherein the first output time period is longerthan the second output time period; outputting each of the plurality ofsub-frames of the viewing image in the set second output time period;and outputting the communication data image in the set first output timeperiod.
 12. A non-transitory computer-readable medium having storedthereon computer-executable instructions that, when executed by areceiving apparatus, cause the receiving apparatus to executeoperations, the operations comprising: capturing a projected image,wherein the projected image is projected by a projector, wherein theprojected image includes a communication data image and a viewing image,the viewing image comprises a plurality of sub-frames, the projectorswitches between the plurality of sub-frames within one image frameoutput time period to output the viewing image; output of thecommunication data image is continuous for the one image frame outputtime period, each bit value of the communication data image is set basedon each configuration bit value of a plurality of configuration bitvalues of communication data, the set each bit value of thecommunication data image is one of equal or inverse of each bit value ofa last sub-frame of the plurality of sub-frames, and the last sub-frameimmediately precedes output of the communication data image; detectingan event corresponding to a luminance change equal to or greater than aspecific threshold; outputting event information including a pixelposition and occurrence time of an event occurrence pixel; receiving theevent information; determining that event occurrence interval of theevent matches the one image frame output time period; and detecting thecommunication data image included in the projected image based on thedetermination; and acquiring the communication data from thecommunication data image based on the plurality of configuration bitvalues of the communication data image.